Incomplete 3D Representation of Video Objects for Multiview Applications

This paper introduces a new form of representation for three-dimensional video objects. We propose to provide the capability to encode depth data for video object planes, as they are used in the MPEG-4 video standard. We introduce a technique to extract depth and texture data from video objects, that are captured with multiple-camera configurations. The video object plane then represents an unwrapped surface of a 3D object, which contains all texture data visible from any of the cameras. Using the depth data, it is possible to reconstruct different viewpoints from this texture surface. The technique can be used for efficient multiview encoding of video objects and for viewpoint adaptation towards single video objects, e.g. in mixing natural and synthetic images. Furthermore, scene composition becomes more flexible if depth information can be used for a natural background scene as well.